<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
 <meta>
  <!-- Stylesheets -->
  <link href="../web.css" type="text/css" rel="stylesheet"></link>
  <link href="../pygmentize.css" type="text/css" rel="stylesheet"></link>
  <title>VLFeat - Documentation - Matlab API - MISC - VL_PEGASOS</title>
  

  <!-- Scripts-->
  
 </meta>

 <!-- Body Start -->
 <body>
  <div id="header">
   <!-- Google CSE Search Box Begins -->
   <form action="http://www.vlfeat.org/search.html" method="get" id="cse-search-box" enctype="application/x-www-form-urlencoded">
    <div>
     <input type="hidden" name="cx" value="003215582122030917471:oq23albfeam"></input>
     <input type="hidden" name="cof" value="FORID:11"></input>
     <input type="hidden" name="ie" value="UTF-8"></input>
     <input type="text" name="q" size="31"></input>
     <input type="submit" name="sa" value="Search"></input>
    </div>
   </form>
   <script src="http://www.google.com/coop/cse/brand?form=cse-search-box&amp;lang=en" xml:space="preserve" type="text/javascript"></script>
   <!-- Google CSE Search Box Ends -->
   <h1><a shape="rect" href="../index.html" class="plain"><span id="vlfeat">VLFeat</span><span id="dotorg">.org</span></a></h1>
  </div>
  <div id="headbanner">
   Documentation - Matlab API - MISC - VL_PEGASOS
  </div>
  <div id="pagebody">
   <div id="sidebar"> <!-- Navigation Start -->
    <ul>
<li><a href="../index.html">Home</a>
</li>
<li><a href="../download.html">Download</a>
</li>
<li><a href="../doc.html">Documentation</a>
<ul>
<li><a href="mdoc.html">Matlab API</a>
</li>
<li><a href="../api/index.html">C API</a>
</li>
<li><a href="../man/man.html">Man pages</a>
</li>
</ul></li>
<li><a href="../overview/tut.html">Tutorials</a>
</li>
<li><a href="../applications/apps.html">Applications</a>
</li>
</ul>

   </div> <!-- sidebar -->
   <div id="content">
    <div class="mdoc">
<ul class="breadcrumb"><li><a href="mdoc.html">Index</a></li><li><a href="VL_OVERRIDE.html">Prev</a></li><li><a href="VL_SAMPLINTHIST.html">Next</a></li></ul><div class="documentation"><p>
W = <a href="VL_PEGASOS.html">VL_PEGASOS</a>(X, Y, LAMBDA) learns a linear SVM W given training
vectors X, their labels Y, and the regularization parameter LAMBDA
using the PEGASOS [1] solver. The algorithm finds a minimizer W of
the objective function
</p><pre>
  LAMBDA/2 |W|^2 + 1/N SUM_i LOSS(W, X(:,i), Y(i))
</pre><p>
where LOSS(W,X,Y) = MAX(0, 1 - Y W'X) is the hinge loss and N is
the number of training vectors in X.
</p><p>
ALGORITHM. PEGASOS is an implementation of stochastic subgradient
descent. At each iteration a data point is selected at random, the
subgradient of the cost function relative to that data point is
computed, and a step is taken in that direction. The step size is
inversely proportional to the iteration number. See [1] for
details.
</p><dl><dt>
NumIterations
<span class="defaults">[10 / LAMBDA]</span></dt><dd><p>
Sets the maximum number of iterations.
</p></dd><dt>
BiasMultiplier
<span class="defaults">[0]</span></dt><dd><p>
Appends to the data X the specified scalar value B. This
approximates the training of a linear SVM with bias.  The bias
can be recovered from the optimal weight vector W as W(end) *
B. It is often useful to precondition the computation of the
gradient with respect to this element by a small value,
e.g. 0.1/B (see PRECONDITIONER).
</p></dd><dt>
StartingModel
<span class="defaults">[null vector]</span></dt><dd><p>
Specify the initial value for the weight vector W.
</p></dd><dt>
StartingIteration
<span class="defaults">[1]</span></dt><dd><p>
Specify the iteration number to start from. The only effect
is to change the step size, as this is inversely proportional
to the iteration number.
</p></dd><dt>
Permutation
<span class="defaults">[empty]</span></dt><dd><p>
Specify a permutation PERM to be used to sample the data (this
disables random sampling). Specifically, at the T-th iteration
the algorithm takes a step w.r.t. the PERM[T']-th data point,
where T' is T modulo the number of data samples
(i.e. MOD(T'-1,NUMSAMPLES)+1). PERM needs not to be
bijective. This allows specifying certain data points more or
less frequently, implicitly increasing their relative weight in
the error term. A common application is to balance an unbalanced
dataset.
</p></dd><dt>
Preconditioner
<span class="defaults">[empty]</span></dt><dd><p>
Specify a diagonal preconditioner PREC. The elements of this
vector are multiplied to the function subgradient before adding
the latter to the current model estimate. The dimension of the
vector PREC is the same of the model, plus one if the SVM has
bias (see BIASMULTIPLIER).
</p></dd><dt>
Verbose
</dt><dd><p>
Be verbose.
</p></dd><dt>
Example
</dt><dd><p>
The options StartingModel and StartingIteration can be used
to continue training. I.e., the command
</p><pre>
  vl_twister('state',0) ;
  w = vl_pegasos(x,y,lambda,'NumIterations',1000) ;
</pre><p>
produces the same result as the sequence
</p><pre>
  vl_twister('state',0) ;
  w = vl_pegasos(x,y,lambda,'NumIterations',500) ;
  w = vl_pegasos(x,y,lambda,'NumIterations',500, ...
                 'StartingIteration', 501, ...
                 'StartingModel', w) ;
</pre></dd><dt>
REFERENCES
</dt><dd><p>
[1] S. Shalev-Shwartz, Y. Singer, N. Srebro, and
A. Cotter. Pegasos: Primal Estimated sub-GrAdient SOlver for
SVM. MBP, 2010.
</p></dd></dl><p>
See also: <a href="VL_HOMKERMAP.html">VL_HOMKERMAP</a>(), <a href="VL_HELP.html">VL_HELP</a>().
</p></div></div>
   </div>
   <div class="clear">&nbsp;</div>
  </div> <!-- pagebody -->
  <div id="footer">
   &copy; 2007-12 Andrea Vedaldi and Brian Fulkerson
  </div> <!-- footer -->

  <!-- Google Analytics Begins -->
  <script xml:space="preserve" type="text/javascript">
   //<![CDATA[
    var localre = /vlfeat.org/;
    if(document.location.host.search(localre) != -1)
    {
   var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
   document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
   }
   //]]>
  </script>
  <script xml:space="preserve" type="text/javascript">
    //<![CDATA[
    var localre = /vlfeat.org/;
    if(document.location.host.search(localre) != -1)
    {

   try {
   var pageTracker = _gat._getTracker("UA-4936091-2");
   pageTracker._trackPageview();
   } catch(err) {}

   }
   //]]>
  </script>
  <!-- Google Analytics Ends -->
 </body>
</html>

 